Camera and scanning technology allows a user to scan a three-dimensional (3D) object and to create a digital 3D model of the scanned object. Handheld scanning technology may allow a user to generate a digital 3D model by moving the scanning device around a target object by, for example, walking in a circle around the object. The digital model may be generated and displayed to the user. In some cases, the digital model includes color or simulated texture, such as by applying a pattern of color to the model.
The quality of such 3D models is partially dependent on the reproduction of small-scale details on the target object. Although reproduction of such minute details is limited by the resolution of the scanning equipment, high-quality color or textures can be simulated on a scanned model. For example, textures can be simulated by applying a pattern of color to a 3D model. Such colors can improve the perceived “realism” of the displayed model, thereby improving the aesthetic qualities of the model.
To create a 3D model, current scanning technologies use depth sensors, such as laser range finders or biopic camera arrangements. Color is added to the model by reconstructing the color of the scanned object. For example, a red-green-blue (RGB) camera or a high-resolution color camera is used with a depth sensor when capturing images used to generate a 3D model. The color images are correlated to the depth images by estimating a transformation matrix between the color camera and the depth camera. Thus, each point in the digital model is correlated with one or more color images, and a color for the point can be determined based on the correlated color images.
Despite recent advances in handheld scanning technology, an effective resolution of handheld scanners is negatively impacted by factors such as device constraints (e.g., size or weight), natural shaking motions of the user holding the equipment, and fluctuations in light and camera position as a user moves the scanner around the object. Furthermore, a user may move the scanning device too quickly, which causes blur in the resulting color images. These factors may result in wide variations in the quality of the color images that are correlated with a given point in the 3D model. These variations in quality complicate the determination of a “correct” color for the given point.
One existing approach to determine the color for a point is to compute an average color from the images (or a subset of the images) in which the point is visible. But the effectiveness of this solution may be limited by wide variations in quality for the color images that result from the use of handheld scanners. For example, some color images may have a poor quality, such as blurriness, low focus, low light, or other indications of poor image quality. Therefore, averaging low-quality images with higher-quality images may result in an incorrect color selection for one or more points, reducing the color fidelity for the scanned 3D model.
Another existing approach is to select a “best quality” image to provide the colors for multiple points. However, an image may have inconsistencies in the quality of different image patches. For example, a patch in one area of an image could have a poor quality, while a patch in another area of an image could have a high quality. Selecting this image as a “best quality” image could result in color selection based on the patches in the poor quality area. Thus, attempts to select a “best quality” image can result in incorrect color selection for some of the points, and can result in poor quality color for the scanned 3D model.
In addition, existing approaches may not account for the number of images used to provide the colors for multiple points. Assigning colors to a model from multiple images may cause seams, or visible edges, to be displayed between points in the model. A 3D model that has colors assigned from a large number of different images can have a large number of a seams between points, creating a “jagged” appearance when the model is displayed to a user. Thus, attempts to assign colors that do not address the number of images used can result in low-quality models with a high number of seams.
Thus, existing solutions may provide low-quality 3D models for reasons such as (but not limited to) those described above.